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■ ABSTRACT 

Present healthcare system calls for standardization and interoperability of images acquired 
by various equipment manufacturers. The healthcare system is also not limited to a single super- 
specialty hospital where state of the art machines and excellent physicians are available but should 
also these services need to be available in remote places too. Nowadays, number of images acquired 
by the hospitals is increasing due to the various modalities available to identify accurately the cause of 
the deceases. This results in more number of medical images being stored in the database in hospitals. 
The interpretation of these medical images becomes even more critical, as it may contain the 
information which cannot be perceived by human eye. Therefore the images need to be annotated 
appropriately while it is stored so that, retrieval of the images of interest can be done quickly. The 
manual annotation is very cumbersome process when large number of images is involved in the 
retrieval process. Hence it is necessary to have an efficient Content Based Image Retrieval System 
which is user friendly and user definable in terms of speed and Precision. 
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I. INTRODUCTION 

All human beings have the inherent nature of organizing the objects based on their perception. When 
we have very few items to arrange or organize, we can do so manually with ease. When the number increases, 
we need the help of an intelligent machine to do the same. In case of non-medical images variation in perception 
can be acceptable, since it may not have major impact even if there is some error. However in case of medical 
images a small error may have serious implications, which should not be overlooked. Hence it is important to 
compare medical images "based on the content" rather than only on perception. Looking for the contents which 
are not visible to the human eye calls for extracting features of the image, storing it in the database and 
retrieving based on the query. 

Figure 1 describes the general block diagram of Image Retrieval System where, features extracted from 
query image are compared with the features in the image database. Pre calculating the features of the images in 
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the image database reduces the time required for comparison and retrieval. However the challenge lies in finding 
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the right features of the image so that, retrieved image is accurate and the retrieval time is also less. User 
authentication is needed to restrict only the authorized user to use the system. The modality selection helps to 
categorize correct modality images to be compared for retrieval. Most of the times, it may not be sufficient to 
have one feature comparison to retrieve the image accurately. It may be necessary to refine the query process by 
extracting more than one feature (for example intensity, shape, edge, texture) so that, retrieved image is more 
precise. 

As the medical images in the PACS system at present need to conform to the DICOM (Digital Imaging 
and Communication in Medicine) standard, the size of any medical image is well defined by National Electrical 
Manufacturer's Association (NEMA) specification. The medical equipment used in the imaging process in 
hospitals or diagnostic centers need to confirm to standards defined by NEMA. 

In order to provide efficient patient care many hospitals are implementing Picture Archival and 
Communication Systems (PACS). In a practical scenario mid-sized hospitals cannot afford to have the PACS 
due to the initial investment which is quite huge. These hospitals can have the services of PACS through a Wide 
Area Network (WAN). Size of the medical image being very large, transmission of these images to a distant 
healthcare setup will be limited by the bandwidth of the Wide Area Network or Internet. The image viewing at a 
remote station will be a very slow process and likely to cause severe delay which is a great concern for a 
physicians if he/she wants to view these images before treatment. Hence it is necessary to have an efficient 
Content Based Image Retrieval System which is user friendly and user definable in terms of speed and 
Precision. 

II. SIGNIFICANCE OF CBIR IN HEALTHCARE 

Images in medical domain are being captured only when the exact diagnosis need to be carried out. The 
images are very rich in their content and the information may not be clearly visible to physicians for visual 
inspection. These images are being large in size and require accurate interpretation a good Content Based Image 
Retrieval system will definitely help physicians in their decision by comparing it with similar cases in the image 
database. 

Many Image Retrieval systems that have been developed are independently accessed and are usually text based. 
These systems have been developed mainly to categorize pictures which are generally non - medical. Here 
image perception is of higher priority than the content and thus text based retrieval is appropriate. 
Apart from the rich information contained in the medical images, medical images also contain meta data 
information such as image modality, image slice number, patient age, gender, family status. Use of meta data 
along with the content of the image will prove to be more accurate in medical image retrieval, than using 
content alone. 

Much of the radiological practice is currently not based on the quantitative image analysis, but on heuristics. 
Such heuristics may fail some time in variety of circumstances such as poor quality of imaging equipment, 
associated noise during the imaging procedure, lack of experienced technician, poor quality imaging station. 
Computer assisted image interpretation and retrieval will assist radiologists in correct interpretation. Image 
retrieval system combined with decision support system will be a real advantage in proper diagnosis. 

There is a need to address the challenge of a specialized internet based Content Based Medical Image 
Retrieval (CBMIR) system that can help the doctors or medical practitioners across the globe in referring the 
existing medical records before taking a final decision over the diagnosis. If the system is distributed and 
accessible to the physicians, there will be definitely improvement in the quality of healthcare. 

III. CBIR SYSTEM EVALUATION 

In the general image retrieval domain, it is difficult to compare any two retrieval systems. Still, there are several 
articles on the evaluation of imaging systems in medicine or on general evaluation of clinical systems. 
Henning Muller [1] provides an overall view of the various CBIR systems being developed. It would be hard to 
name or compare them all but some well-known examples include Candid, Photobook and Netra that all use 
simple color and texture characteristics to describe the image content. Using higher level information, such as 
segmented parts of the image for queries, was introduced by the Blob world system. PicHunter on the other hand 
is an image browser that helps the user to find an exact image in the database by showing to the user images on 
screen that maximizes the information gain in each feedback step. A system that is available free of charge is the 
GNU Image Finding Tool (GIFT). Some systems are available as demonstration versions on the web such as 
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Viper, WIPE or Compass. 

A single example result does not reveal a great deal about the real performance of the system and is not 
objective as the best possible query result can be chosen arbitrarily by the authors. 
The commonly used measurement parameters in CBIR are: 

1. Retrieval Time (tr): The time taken by the system to display relevant images to the given query image 

number of relevant [terns retrieved 

2. Precision (Pj = ■ 

number or items retrieved 

number of relevant items retrieved 

3. Recall (R] = ; — ; 

number of relevant items 

Defining the relevant image set can be done in various ways. This paper defines a set of "n" images which are 
varying by some factor as defined by the user. A total of 1720 images procured from Radio diagnosis 
department of K.M.C Manipal hospital is maintained in the database. The images are of different modality like 
CT, MR, US & CR and of dimension (256 x 256, 2500x2048, etc..) 

Figure 2 shows the relevant set for the query image 14. 
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IV. RETRIEVAL METHODS IMPLEMENTED 

Retrieval time (tr) comparison of eight different methods is provided in section 4.6 in the form of a graph. 

4.1 Histogram Based Retrieval 

One of the most commonly used parameters for image comparison is the color or intensity of the 
image. In our research we have emphasized on the gray level medical images. An experienced physician can 
interpret the images better than a lay man due to his experience in identifying similar images in the past. 
However subtle changes in the image cannot be easily observed. A histogram is the count of the number of 
pixels at each intensity level over the entire image. It is given by, 

hist(r k ) = n k 

Where, k=0 ...L-l, 

L is the number of intensity levels. 

nk = number of pixels at gray level rk 

It plots the number of pixels for each intensity value. By looking at the histogram for a specific image, 
a viewer will be able to judge the entire intensity distribution at a glance. 

In histogram matching technique, the histogram of all the images in the database are computed and 
stored in a file. When a query image is given, the histogram of the query is computed and compared with the 
stored histograms. For each and every image in the database, the distance metric is calculated as, 

255 

histdist[dataset] = ^1 hist _ dataset[j] — hist _ query[j] I 

7=0 

Where, 

o j denotes the various gray levels ranging from to 255. 

o hist_query is the histogram of query image, hist_dataset is histogram of the image in the 
database. 

The images in the database nearly matching with the query image, have the least distance metric. The exact 
match is the one with the zero distance metric. 
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Figure 3 



Images in the database 



The result of the Histogram based retrieval is shown in Fig. 3. Twelve closest images are displayed from a total 
1720 images in the database. The total time for retrieval is 31.5503 seconds. 

4.2 Bit Plane Based Retrieval 

In this method, the image is composed of eight one bit planes, ranging from bit plane for the Least 
Significant Bit (LSB), to the bit plane 7 for Most Significant Bit (MSB) as shown in Fig.4. Usually most of the 
visually significant information is contributed by higher order bits and less by the least significant bits. An 8 bit 
image can have 256 different shades of gray, where represents black and 255 represent white, and Integers 
between and 255 represent various shades of gray. So out of the 8 bits, when 7th bit is set, it represents the 
intensities ranging from gray level 128 to 255, and, when 6th bit is set, it represents totally 128 gray levels in the 
range 64 to 127 and 192 to 255, and when 5th bit is set, represents totally 128 gray levels in the range 32 to 63; 
96 to 127; 160 to 191 and 224 to 255 and so on. 



One S-bit pixel value 



Bit Plane? 
(MSB) 



Bit Plane 
(LSB) 




In this method, the number of occurrences of set bits in each bit plane is computed. The bit plane histogram is 
represented mathematically as, 



m,n 



hist _ databaset\i\ = 




f(x 9 y)i 



(2.5) 



th 



x,y 



Where, f(x, y) is the pixel value at (x, y) having i bit 



set to 1, i = 7,6. .0. 
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4.3 Hierarchical Bit Plane based retrieval 

To make the bit plane histogram matching more efficient, a method is proposed to discard the 
dissimilar images in the database by a method called Hierarchical bit plane histogram matching. 

4.3.1 Histogram percentage as threshold 

This method begins with the most significant bit (MSB), and is based on the distance metric equation. 
The distance metric at the MSB for each image in the database is compared with the threshold specified by the 
user in terms percentage of histogram. Those images in the database with the distance metric less than the 
specified threshold are retained for subsequent search at lower bit planes. This is continued till the least 
significant bit i.e. till the Oth bit is reached. This bit plane technique allows one to compare the query and 
database images with only one arithmetic operation per bit plane and hence needs less computational time and 
power compared to the traditional grayscale histogram matching. 

4.3.2 Image size as the threshold 

It is common that, the major object in an image is located at the center part of the image. In this method 
query image as well as the images in the database are re-sized according the percentage of the threshold 
specified by the user in terms of dimension. If image dimension is [m x n] and the percentage of the threshold is 
90%, then image dimension is re-sized to 90% * [m x n]. it has been observed that smaller dimension of the 
image takes less time for retrieval. 



Result of Histogram and Bit plane 

Table 1, shows comparison of four retrieval methods in terms of retrieval time, Precision and Recall 
parameters. 



Table 1 



Total number of images in the database= 1720 


Sl.No 


Retrieval Method 


Retrieval time (Tr) in sees 


Precision (P) 


Recall (R) 


1. 


Histogram comparison 


31.6323 


1 


0.667 


2. 


Bit plane histogram 


4.27613 


0.583 


0.389 


3. 


Hierarchical bit plane 2% histogram as the threshold) 


4.64211 


1 


0.667 


4. 


Hierarchical bit plane 98% size of the image as the threshold) 


5.31345 


1 


0.667 
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Fig. 5, Retrieval of four different methods 
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Fig. 6, Precision, Recall graph of four methods 

From the graph in the Fig. 5, it can be observed that, there is a considerable improvement in retrieval 
time in case of Bit plane histogram compared to that Simple histogram method. Fig. 6 shows the Precision and 
Recall parameters of the four different methods implemented. From the above two graphs it can be observed 
that the least retrieval time (Tr) amongst the four method is the Bit plane histogram (4.27613 sec). While 
retrieval time being the lowest in Bit plane method, the Precision and Recall values are 0.583 and 0.387 
respectively, which is lower than the other 3 methods namely Histogram comparison, Hierarchical bit plane with 
Histogram as threshold and Hierarchical bit plane with size as the threshold. From the user's perspective the 
Precision and Recall being the most important parameters, Hierarchical bit plane with 2% histogram as the 
threshold is the best method amongst the four being described here. It is also observed that the Precision and 
Recall parameters remain same irrespective of the image dimension in the database. 

The retrieval time will also depend on the speed and type of the processor on which the algorithm has 
been implemented. Results are obtained on a computer with Processor type: Intel i3-2370M CPU @ 2.4 GHz 
processor with 4 GB RAM. 

4.4 Cumulative Distribution Function (CDF) based retrieval 

Medical images are of different dimension depending on the modality of the image as defined by 

Radiological Society of North America (RSNA). The pixel depth and image dimension vary depending on the 
size of the image. The comparison of images of different dimension will be difficult and needs sophisticated 
algorithms to match the two images in terms of intensity or histogram value. This puts extra computation time as 
well as complexity to the CBIR system. Another problem being the registration of the two images. In the pixel 
matching method where pixel intensity of the query and database images is compared, the rotation of the image 
leads to comparison of two dissimilar pixels. This requires image registration to be carried out before the 
comparison. This problem is overcome by CDF where The CDF of the query image and the images in the 
database are approximated by piecewise linear models with two parameters, slope and intercept at various 
grayscale intervals. The contiguous set of lines approximating the CDFs enables us to compare the query image 
and the images in the database with corresponding estimated slopes and intercepts. As the dynamic range of 
CDF is from to 1, images of different sizes can be compared. Approximation of CDFs with lines further 
reduces the dimension of the image features and thus improves the speed of matching. Also, the monotonically 
increasing CDF is well suited for approximations with lines. Resolving the CDF with lines of different lengths 
recasts the matching to a hierarchical methodology. 

Cumulative Distribution Function (CDF) for retrieving the medical images from the database provides 
a considerable reduction in the retrieval time and also flexibility to the user in terms of selecting suitable number 
of line segments (p) and the percentage of CDF threshold depending on the situation providing control over 
precision and time of retrieval. 
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The Cumulative Distribution Function cdf (i) upto the gray level T is given by: 
i i 

cdfti) = ^ h(j) = 2 ^- < i < 255 (3.2] 

Where, h (j) is the normalized histogram at gray level j 
nj is the number of pixels with gray level j 
M.N is the size / dimension of the image 

The CDF contains the same information as that of the histogram, but in another form. However, it has 
two interesting properties (a) It is a monotonically increasing quantity which allows fairly well approximation of 
the CDF with just a first order curve (b) It has a dynamic range between and 1 which allows one to fit 
piecewise linear models on CDFs of images of any size. 
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Fig 7. CDF of image brain0500.jpg with p=4 and continuous cdf overlapped 



4.5 Edge and Texture based retrieval 

It is also equally important to consider the edge / shape of the medical images for comparison 

especially in case of bone / skull images. As the edges give idea about the shape of the objects present in the 
image, they are useful for segmentation, registration and identification of objects. We have implemented edge 
detection using Sobel Operator. The Sobel operator consists of a pair of 3x3 convolution kernels as shown in 
Fig. 8. One kernel is simply the other rotated by 90°. Sobel operator is very simple and effective way for finding 
the edges in image. 
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These kernels are designed to respond maximally to edges running vertically and horizontally relative 
to the pixel grid, one kernel for each of the two perpendicular orientations. The kernels can be applied separately 
to the input image, to produce separate measurements of the gradient component in each orientation (call these 
Gx and Gy). These can then be combined together to find the absolute magnitude of the gradient at each point 
and the orientation of that gradient. An approximate gradient magnitude is computed using 



The Edge = IGd - Gql 

Where Gd is the gradient of the database image 
Gq is the gradient of the query image 

Texture is a very general notion that can be attributed to almost everything in nature. For a human, the 
texture relates mostly to a specific, spatially repetitive (micro) structure of surfaces formed by repeating a 
particular element or several elements in different relative spatial positions. Generally, the repetition involves 
local variations of scale, orientation, or other geometric and optical features of the elements. Texture features 
can be extracted in several methods, using statistical, structural, model based and transformation based, in which 
the most common way is, using the Gray Level Co- occurrence Matrix (GLCM). GLCM contains the second- 
order statistical information of spatial relationship of pixels of an image. From GLCM, many useful textural 
properties can be calculated to expose details about the image content. However, the calculation of GLCM is 
very computationally intensive and time consuming. After calculating GLCM, normalization is being carried out 
by dividing each element by the sum of all elements as shown in the equation below: 



From the normalized GLCM the following features have been extracted 

• Contrast [Tl] 

• Dissimilarity [T2] 

• Homogeneity [T3] 

• Angular Second Moment [T4] 

• Entropy [T5] 

The calculation of GLCM features is done by the following equation: 



\G\ = \Gx\ + \Gy\ 




ij=0 



Where, 



Vi,j is the (i , j) element of the GLCM 

Pi,j is the (i , j) element of the Normalized GLCM 




A'-l 



Dissimilarity [T2] = 




t\/=0 
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From the above equations the feature vector F is calculated as shown below. 

F= [T1,T2, T3,T4, T5] 
The magnitude of the feature vector F = T1+T2+T3+T4+T5 
The Similarity metric of the texture can be calculated as Etexture = I Fd - Fq I 
Where, 

• Fd = Magnitude of Feature Vector of Database Image 

• Fq = Magnitude of Feature Vector of Query Image 

The results of our research work where we have implemented eight different methods are shown in are shown in 
Table 2. 

4.6 Retrieval time comparisons of various methods implemented 



Sl.No 


Method 


Retrieval Time 
(Tr) in seconds 


1. 


Histogram 






31.6323 


2. 


Bit plane 


Simple 




4.09868 


3. 


Hierarchical bit plane -1 


20% histogram as threshold 




4.51706 


4. 


Hierarchical bit plane -2 


98% size as threshold 




5.69741 


5. 


CDF 


P = 3 [8 lines] 




0.61504 


6. 


Hierarchical CDF 


P = 3 


1 % of cdf diff as threshold 


0.351143 


7. 


Edge based - 1 


Sobel method 




12.6268 


8. 


Texture based 


GLCM method 




0.401352 



Table 2 
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Fig. 9 
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Fig. 10. Retrieved images for query image brain0014.jpg with p=8, with 1% as CDF threshold 



Figure 9 shows the retrieval time (tr) comparison of various methods implemented in our research 
work. Y- axis indicates the retrieval time (tr) in seconds. Figure 10 shows the screen shot for CDF based 
retrieval with p=8 with 1% as CDF as the threshold. 



X - axis indicates 8 different methods implemented. The details of the 8 different methods are given below: 

1 . Histogram based retrieval (without pre-stored features) 

2. Simple Bit plane (pre-stored features) 

3. Hierarchical Bit plane - 20% histogram as the threshold (pre-stored features) 

4. Hierarchical Bit plane - 98% size as the threshold (pre-stored features) 

5. CDF based retrieval (p=3) (pre-stored features) 

6. Hierarchical CDF based retrieval (p=3) (pre-stored features) 

7. Edge based retrieval (Sobel method) (pre-stored features) 

8. Texture based retrieval (GLCM method) (pre-stored features) 



V. CONCLUSION AND FUTURE SCOPE FOR THE RESEARCH 

Content-Based Image Retrieval (CBIR) is a very important research area with its innumerable 
applications especially in the medical and healthcare domain. Presently there is a substantial gap between CBIR, 
and its focus on raw image information and decision support systems, which typically enter the workflow 
beyond the point of image analysis itself. This gap represents what we believe is a major opportunity to develop 
decision support systems that integrate image features exploited in CBIR systems. With such integration, CBIR 
may be a starting point for finding similar images based on pixel analysis, but the process would be augmented 
by inclusion of image and non-image metadata as well as knowledge models, broadening the system from 
"image based search" to "case-based" search. 



The present research work focuses on retrieval of images quickly based on three parameters namely 
Intensity, Texture and Shape. However including physician in the image retrieval loop (relevance feedback) will 
be better for accurate diagnosis. It would be better, if the physician is given a choice to select a specific area in 
the image and image is re-sized according to the selected area for better precision. However this will need extra 
time in terms of re-building the feature database. 

The proposed system addresses global feature extraction of the images. However implementing local 
feature extraction based on automatic segmentation of the images may improve the accuracy of the system. 
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In our systems we have maintained single CDF feature database irrespective of line segments (2p) 
being selected by the user. Maintaining different feature databases as per the number of line segment should 
improve the retrieval time. This method could be explored in the future implementations. 

In the texture comparison using GLCM we have taken into account the added values of the features (i.e. Tl 
+ T2 +T3 + T4 + T5). Retrieval performance based on the individual GLCM features may give us more insight 
into the content of the medical images. 

Extending the above features into the video images such as endoscopy may also help physicians in the 
diagnosis. 

From the research perspective it is better to develop a standard CBIR frame work where various algorithms 
can be implemented on standard image database hosted at a central location. Giving options to students / 
researches to develop new algorithms and adding it to the existing methods will make the CBIR system usable 
by everyone. 

REFERENCES 

[1] Henning Muller, Nicolas Michoux, David Bandon, Antoine Geissbuhler. A review of content based image retrieval systems in 

medical applications - Clinical benefits and future directions. International Journal on Medical Informatics (2004) 73, 1-23. 

[I] Manjunath K N, Niranjan U C. Proceedings of the 2005 IEEE. Engineering in Medicine and Biology, 27 th Annual Conference, 
Shanghai, China, September 1 -4, 2005 

[2] Suyog Dutt Jain, Harishchandra Hebbar, K N Manjunath, U.C. Niranjan. Large Scale Distributed Frame work for Remote 

Clinical Diagnosis with Visual Query Support. Distributed Diagnosis and Home Healthcare, Pages: 1-16. ISBN: 1-58883 - 158 - 
2. 

[3] K.N.Manjunath, A Renuka, U.C. Niranjan. Linear Models of Cumulative Distribution Function for Content -based Medical 

Image Retrieval. Journal of Medical Systems (2007) 31:433-443. 
[4] Ritendra Datta, Jia Li, James Z Wang. Content -Based Image Retrieval - Approached and Trends of the New Age. ACM1- 

58113-940-3/04/0010. 

[5] Ceyhun Burak Akgul, Daniel L Rubin, Sandy Napel, Christopher F Beaulieu, Hayit Greenspan, Burak Acar. Journal of Digital 

Imaging, 08 April 2010,Published online 
[6] Datta R, et al: Image retrieval: ideas, influences, and trends of the new age. ACM Computer Survey 40(2), 2008. 

[7] Smeulders AWM, et al: Content-based image retrieval at the end of the early years. IEEE Trans Patt Anal Mach Intell 22 

(12): 1349-1380, 2000. 

[8] Swain, M. J., and Ballard, D. H., "Color indexing for Content Based Image Retrieval" International Journal of Computer Vision 

7(l):ll-32, 1991. 

[9] Comaniciu D, Meer P, Foran DJ: Image-guided decision support system for pathology. Mach Vis Appl 1 1(4):213-224, 1999. 

[10] Kwak DM, et al: Content-based ultrasound image retrieval using a course to fine approach. Ann N Y Acad Sci 980:212-224, 
2002. 

[II] Lim J, Chevallet J-P: Vismed: A visual vocabulary approach for medical image indexing and retrieval, in Second Asia 
Information Retrieval Symposium. 2005. Jeju Island, Korea. 

[12] Shyu CR, et al: ASSERT: a physician-in-the-loop content-based image retrieval system for HRCT image databases. Comput Vis 

Image Underst 75(1/2): 11 1-132, 1999. 
[13] Cauvin JM, et al: Computer-assisted diagnosis system in digestive endoscopy. IEEE Trans Inf Technol Biomed 7 (4):256-262, 

2003. 

[14] Giild MO, et al: Content-Based Retrieval of Medical Images by Combining Global Features. Accessing Multilingual Information 
Repositories, in Accessing Multilingual Information Repositories. 2005: Springer LNCS 4022. 

[15] Lubbers K, et al: A Probabilistic Approach to Medical Image Retrieval, in Multilingual Information Access for Text, Speech and 
Images. Springer Berlin, 2005, pp 761-772. 



Issn 2250-3005 




November 1 1 2013 



HI 



Page 57 



